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DETAILED ACTION 



Claim Rejections - 35 USC §102 



1. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

2. Claims 1, 2, 5, 6, 7, 14, 15, 16, 17, 18, 20, 21, 24, 25, 37-38, 41-42, and 49-53 are 
rejected under 35 U.S.C. 102(e) as being anticipated by Ellenby (US 6,370,556). 

For claim 1, Ellenby discloses a method of augmenting an image of a scene, comprising: 

obtaining by a first computing unit (Figure 1) a digital image of a scene; 

acquiring by the first computing unit a digital elevation model of the scene, wherein the 
digital elevation model is based on elevation data corresponding to information about the scene 
(Figure 9; Column 7, Lines 25-46); 

registering by the first computing unit the digital image with the digital elevation model 
to create a registered digital image (Figure 9); 

providing the registered digital image from the first computing unit to a second 
computing unit coupled to the first computing unit by a communications network (Column 3, 
Lines 14-46); and 
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augmenting the registered digital image provided to the second computing unit with at 
least some of the information in response to input from the second computing unit ( Column 3, 
Lines 26-37). 



Ellenby does not explicitly teach the registering of the image, but it is inherent in his 
invention that some type of registration or tagging of the image data is required to augment the 
different images into one composite image. 

For claim 2, Ellenby discloses all the previous limitations, wherein obtaining the digital 
image comprises obtaining an image of the scene in real time with a fixed image capturing 
device (Column 2, Line 47). 

For claim 5, Ellenby discloses all the previous limitations, wherein the obtaining 
comprises obtaining an analog image and digitizing the analog image (Column 2, Lines 21-24). 

For claim 6, Ellenby discloses all the previous limitations, wherein the acquiring 
comprises acquiring a preexisting digital elevation model (Figure 9). 

For claim 7, Ellenby discloses all the previous limitations, wherein the information 
comprises a location of at least one above-ground feature within the scene (Figure 1 1). 

For claim 14, Ellenby discloses all the previous limitations, further comprising rendering 
the digital image at the second computing unit to produce a rendered image, wherein the 
augmenting comprises: 

identifying at the second computing a point of interest in the rendered image (Figure 9, 
Element 94); 
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providing the identified point of interest from the second computing unit to the first 
computing unit (Figure 9, Element 94); 

reverse projecting the identified point of interest at the first computing unit to determine 
coordinates thereof (Column 7, Lines 25-60); and 

accessing the information at the first computing unit corresponding to the coordinates 
(Column 7, Lines 25-60). 

For claim 15, Ellenby discloses all the previous limitations, wherein the augmenting 
further comprises overlaying the information corresponding to the coordinates on the registered 
digital image (Figure 9). 

For claim 16, Ellenby discloses all the previous limitations, wherein the information 
comprises textual cartographic data (figure 9). 

For claim 17, Ellenby discloses all the previous limitations, wherein the information 
comprises graphical cartographic data (Figure 9). 

For claim 18, Ellenby discloses all the previous limitations, further comprising rendering 
the registered digital image at the second computing point unit to create a displayed image, 
wherein the input comprises pointing by a user to at least one area of the displayed image (Figure 
9, Element 91). 

For claim 20, Ellenby discloses a method of augmenting an image of a scene, comprising: 
a computer unit programmable for: 

obtaining by a first computing unit (Figure 1) a digital image of a scene; 
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acquiring by the first computing unit a digital elevation model of the scene, wherein the 
digital elevation model is based on elevation data corresponding to information about the scene 
(Figure 9; Column 7, Lines 25-46); 

registering by the first computing unit the digital image with the digital elevation model 
to create a registered digital image (Figure 9); 

providing the registered digital image from the first computing unit to a second 
computing unit coupled to the first computing unit by a communications network (Column 3, 
Lines 55-56); and 

augmenting the registered digital image provided to the second computing unit with at 
least some of the information in response to input from the second computing unit (Column 3, 
Lines 26-37). 

Ellenby does not explicitly teach the registering of the image, but it is inherent in his 
invention that some type of registration or tagging of the image data is required to augment the 
different images into one composite image. 

The unit, which augments the images together, must have been programmed by some 
computer code in order for it to take place. 

For claim 21, Ellenby discloses all the previous limitations, wherein obtaining the digital 
image comprises obtaining an image of the scene in real time with a fixed image capturing 
device (Column 2, Line 47). 

For claim 24, Ellenby discloses all the previous limitations, wherein the obtaining 
comprises obtaining an analog image and digitizing the analog image (Column 2, Lines 21-24). 
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For claim 25, Ellenby discloses all the previous limitations, wherein the acquiring 
comprises acquiring a preexisting digital elevation model (Figure 9). 
For claim 32, refer to claim 14. 
For claim 33, refer to claim 15. 
For claim 34, refer to claim 16. 
For claim 35, refer to claim 17. 
For claim 36, refer to claim 19. 

For claims 37 -38, 41-42, and 49-53, refer to claims 1-2, 6-7, and 14-18. Image 
processing must be done by a computer program or other executable files. Program storage 
devices include floppy disks, CD ROMs, and hard drives etc. 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



4. Claims 3, 9, 10, 12, 22, 27, 28, 30, 39,44,45, and 47 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Ellenby (US 6,307,556). 

For claim 3, Ellenby discloses all the previous limitations, but fails to disclose wherein 
obtaining the digital image comprises obtaining an image of the scene in real time with a mobile 
image capturing device. However, it is notoriously well known that imaging devices can be 
mobile. Mobile imaging devices are able to capture images, which may be moving (Figure 6; 
The helicopter is flying and taking an image). 



Claim Rejections - 35 USC §103 
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Therefore, official notice is hereby given and thus it would have been obvious to one of 
ordinary skill in the art at the time of invention to configure the system of Ellenby with a mobile 
image-capturing device in order to capture images, which may be on the move (Figure 6; The 
helicopter is flying and taking an image). . 

For claims 9 and 10, Ellenby discloses all the previous limitations wherein the obtaining 
comprises obtaining an image of the scene from an image-capture device, the method further 
comprising determining an orientation and positional information for the image capture device 
(Fig. 10; Column 7, Line 62 through Column 8, Line 30). 

However, Ellenby doesn't distinctly show a zoom factor for the images, although it is 
implied in Figure 8. Zoom factors are well known in the art and have been used to place images 
in computer generated images and scenes. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of the invention to include a zoom factor to determine an 
orientation of the image. 

For claim 12, Ellenby discloses all the previous limitations and also wherein the image 
capture device is fixed, and wherein the determining comprises reading positional information 
from the image-capturing device and obtaining the orientation from the equipment positionally 
fixed relative to the image capturing device. 

However, Ellenby doesn't distinctly show a zoom factor for the images, although it is 
implied in Figure 8. Zoom factors are well known in the art and have been used to place images 
in computer generated images and scenes. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of the invention to include a zoom factor to determine an 
orientation of the image. 
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For claim 19, Ellenby discloses all the previous limitations, and also wherein the 
communication means exchange information from other vision systems. It would be obvious to 
one of just ordinary skill in the art at the instant of the invention to use a global communications 
network to allow for information exchange. Motivation would include transferring information 
globally. Official notice is hereby given that transferring information over a global 
communications network is well known. Ellenby teaches the exchange of information form 
different perspectives through communication (Column 2, Lines 55-68). Therefore it would 
have been obvious to one of ordinary skill in the art at the time of the invention to use a global 
information exchange. 

For claim 22, Ellenby discloses all the previous limitations, but fails to disclose wherein 
obtaining the digital image comprises obtaining an image of the scene in real time with a mobile 
image capturing device. However, it is notoriously well known that imaging devices can be 
mobile. Mobile imaging devices are able to capture images, which may be moving (Figure 6; 
The helicopter is flying and taking an image). . 

Therefore, official notice is hereby given and thus it would have been obvious to one of 
ordinary skill in the art at the time of invention to configure the system of Ellenby with a mobile 
image-capturing device in order to capture images, which may be on the move. 

For claims 27 and 28, Ellenby discloses all the previous limitations wherein the obtaining 
comprises obtaining an image of the scene from an image-capture device, the method further 
comprising determining an orientation and positional information for the image capture device 
(Fig. 10; Column 7, Line 62 through Column 8, Line 30). 
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However, Ellenby doesn't distinctly show a zoom factor for the images, although it is 
implied in Figure 8. Zoom factors are well known in the art and have been used to place images 
in computer generated images and scenes. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of the invention to include a zoom factor to determine an 
orientation of the image. 

For claim 30, Ellenby discloses all the previous limitations and also wherein the image 
capture device is fixed, and wherein the determining comprises reading positional information 
from the image-capturing device and obtaining the orientation from the equipment positionally 
fixed relative to the image capturing device. 

However, Ellenby doesn't distinctly show a zoom factor for the images, although it is 
implied in Figure 8. Zoom factors are well known in the art and have been used to place 
images in computer generated images and scenes. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time of the invention to include a zoom factor to determine an 
orientation of the image. 

For claims 39, 44, 45, and 47, refer to claims 3,9,10, and 12. A computer program or 
other executable files must do image processing. Program storage devices include floppy disks, 
CD ROMs, and hard drives etc. 

5. Claims 4, 8, 1 1, 13, 23, 26, 29, 31, 40, 43, 46, and 48 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Ellenby (US 6,307,556) in view of Matsuoka (US 2001/0028350). 

For claim 4, Ellenby discloses all the previous limitations, but fails to disclose wherein 
obtaining the digital image comprises obtaining a preexisting image of the scene. 
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However, Matsuoka discloses a maps database device in which a map is displayed based 
on coordinates of a location disclosed by a GPS sensor. The maps are displayed based on where 
the device is located. The augmented reality vision system synthesizes images based on a 
coordinate system. Configuring the two devices together would allow for the advantage of 
combining preexisting images and for images taken from an image capture unit in real time. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine the systems of Ellenby and Matsuoka in order to achieve an augmented 
reality vision system that synthesizes images in real time with a maps database that could 
synthesize both images together. 

For claim 8, Ellenby discloses all the previous limitations, and Matsuoka discloses 
wherein the information comprises a location of at least one underground feature within the 
scene (Figure 17). 

For claim 11, Ellenby and Matsuoka disclose all the previous limitations and also 
wherein the positional information comprises latitude, longitude (Figure 2; Matsuoka), and 
elevation above ground (Figure 9; Ellenby). 

For claim 13, Ellenby and Matsuoka disclose all the previous limitations and also 
wherein the equipment comprises a global positioning system (Matsuoka; Figure 1, Element 1C), 
a digital compass (Matsuoka; Figure 1 1) and a digital inclinometer (Matsuoka; Figure 13). 

For claim 23, Ellenby discloses all the previous limitations, but fails to disclose wherein 
obtaining the digital image comprises obtaining a preexisting image of the scene. 

However, Matsuoka discloses a maps database device in which a map is displayed based 
on coordinates of a location disclosed by a GPS sensor. The maps are displayed based on where 
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the device is located. The augmented reality vision system synthesizes images based on a 
coordinate system. Configuring the two devices together would allow for the advantage of 
combining preexisting images and for images taken from an image capture unit in real time. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to combine the systems of Ellenby and Matsuoka in order to achieve an augmented 
reality vision system that synthesizes images in real time with a maps database that could 
synthesize both images together. 

For claim 26, Ellenby discloses all the previous limitations, and Matsuoka discloses 
wherein the information comprises a location of at least one underground feature within the 
scene (Figure 17). 

For claim 29, Ellenby and Matsuoka disclose all the previous limitations and also 
wherein the positional information comprises latitude, longitude (Figure 2; Matsuoka), and 
elevation above ground (Figure 9; Ellenby). 

For claim 31, Ellenby and Matsuoka disclose all the previous limitations and also 
wherein the equipment comprises a global positioning system (Matsuoka; Figure 1, Element 1C), 
a digital compass (Matsuoka; Figure 1 1) and a digital inclinometer (Matsuoka; Figure 13). 

For claims 40, 43, 46, and 48, refer to claims 4, 8, 1 1, and 13. A computer program or 
other executable files must do image processing. Program storage devices include floppy disks, 
CD ROMs, and hard drives etc. 

Conclusion 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gary L Solomon whose telephone number is (703)-305-4370. 
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7. The examiner can normally be reached on Monday - Friday 8:00 AM - 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's primary, Vu Le 

can be reached on (703)-3 08-66 13. 

Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

Or faxed to: 

(703) 872-9314, (for informal or draft communications, please label 
"Proposed" or "Draft") 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, Arlington, 
VA., Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application should be 
directed to the customer service number (703) 306-0377. 
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